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TITLE OF THE INVENTION 



USER INTERFACE DISPLAY APPARATUS AND METHOD OF 
CONTROLLING SAME 

5 

BACKGROUND OF THE INVENTION 

This invention relates to a user interface display 
apparatus for operating, e.g., an image processing 
10 apparatus, and to a method of operating the image 
processing apparatus, for the purpose of enhancing 
operability . 

In an information processing apparatus such as a 
personal computer (referred to as a "PC" below) , an 

15 image processing apparatus such as a printer or 

facsimile machine connected to the PC and application 
software (referred to simply as an "application" below) 
for operating the image processing apparatus are used to 
display a user interface on the PC. The user interface 

2 0 on the PC is operated to utilize the functions of the 
image processing apparatus . In a typical example of 
such an arrangement, binary or text data that has been' 
stored in a storage device, e.g., a memory or a hard 
disk drive (HDD) within the PC can be transmitted by the 

25 connected facsimile machine using a facsimile 

application, and data that has been received by the 

- 1 - 



facsimile machine can be displayed on a display device, 
which has been connected to the PC, using the facsimile 
application. 

Image data to be transmitted for facsimile purposes 
5 can also be entered from an image scanner or the like 
connected to the PC. In this case the procedure 
involves the user loading the document in the image 
scanner, launching the image reading application to read 
the image from the document, storing the read image data 
10 temporarily on a storage device within the PC (or in an 
external storage device accessible from the PC) and then 
launching the facsimile application to transmit the 
image data. 

Similarly, in a case where a document copying 
15 operation is performed using an image scanner and a 
printer that are connected to a PC, the procedure 
followed involves the user launching a copying 
application to read the image of the document, input the 
image to the PC and then output the image from the PC to 
2 0 the printer, as described in the specification of 

Japanese Patent Application Laid-Open (KOKAI) No. 9- 
288554, by way of example. On the other hand, an 
embodiment in which the fact that a document has been 
placed in a scanner is sensed, in response to which the 
25 document is read, is described in the specification of 
Japanese Patent Application Laid-Open No. 9-93381. 



However, in a case where the image processing 
apparatus connected to the PC in the examples of the 
prior art mentioned above is an apparatus capable of 
being used especially as a stand-alone device, i.e., in 
5 a case where the apparatus is a copier or facsimile 
machine, the control panel provided on the image 
processing apparatus so that it may be used in stand- 
alone fashion and the image which the PC displays on the 
display device when data is sent and received are 

10 completely different. 

For example, a command for implementing a function 
which is a button on the control panel of a facsimile 
machine is displayed only as characters on the display 
device of the PC. Even if the command is displayed 

15 graphically, the design and placement of the graphic 

display differ entirely from the button on the control 
panel. As a consequence, this approach does not allow 
one to intuit, in terms of appearance, the association 
between the display and the control panel of the actual 

20 facsimile machine. 

Thus, the control panel possessed by the image 
processing apparatus itself and the control-panel 
display presented on the PC connected to this image 
processing apparatus are visually quite different from 

25 each other and there is no clear association between the 
display and the actual apparatus . This leads to 



inconveniences. For example, consider a case where the 
method of operation i*sed when making a transmission 
differs depending upon whether the information medium is 
tangible (information on paper) or intangible 
5 (electronic information) , as when information that has 
been written on paper is transmitted by operating a 
facsimile machine or when electronic data, such as a 
word-processed document or an image created by a PC, is 
transmitted from the PC by operating the facsimile 

10 machine. In such case the operation performed at the 
actual apparatus (the facsimile machine) and the 
operation performed at the PC are different despite the 
fact that the same apparatus is operated. This not only 
makes it necessary to remember both methods of operation 

15 but also is a cause of erroneous operation. 

As another example, consider a case where a 
plurality of types of image processing apparatus having 
the same functions are capable of being utilized by a 
PC. In this case also the PC displays only the same 

2 0 operating screen, making it difficult for the operator 
to tell which image processing apparatus is being 
instructed to perform processing. For example, in a 
situation where both a color copier and a black-and- 
white copier are capable of being used, the operator 

2 5 selects the particular output destination depending upon 
the type of document that is to be output. With the 
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prior art, however, the screen display presented by the 
PC does not differ that much, the display differing in 
terms of the model number of the image processing 
apparatus at best. This makes it easy for the operator 
5 to perform an erroneous operation, such as accidentally 
instructing that a color document be output to the 
black-and-white copier. 

Similarly, consider application software run by the 
PC to utilize a remote image processing apparatus. The 

10 screen display and the method of operation differ for 
each application, and even if the application is the 
same, version updates can require a change in the method 
of operation. As a result, the environment to which the 
operator has grown accustomed must be altered on each 

15 occasion. 

Further, in a case where an image that has been 
entered from an image input apparatus connected to the 
PC is to be transmitted for facsimile purposes or 
copied, the operation for reading the document must be 

2 0 performed after first launching not only the facsimile- 
transmission application and copying application but 
also the application that is for reading the image. If 
the PC has not been started up or has been placed in a 
power conserving mode, for example, the user must 

2 5 perform all of the above-mentioned operations, from 

activation of the PC to launching of the applications. 
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This is a more troublesome and time-consuming operation 
than what would be entailed by using the devices such as 
the facsimile machine and copier in stand-alone fashion. 

5 SUMMARY OF THE INVENTION 

Accordingly, an object of the present invention is 
to provide a remotely controllable apparatus and remote 
control method in which there is little disparity 
10 between a case where the apparatus is controlled using a 
control panel possessed by the apparatus itself and a 
case where the apparatus is controlled remotely from a 
PC or the like, thereby facilitating control of the 
apparatus . 

15 . Another object of the present invention is to 

provide a method of remotely controlling a remotely 
controllable apparatus in which operability is enhanced 
when remotely controlling a data processing apparatus 
from a PC. 

20 A further object of the present invention is to 

enhance convenience when transmitting or outputting the 
image of a document utilizing an image input device and 
a remotely controllable apparatus connected to a PC. 
Specifically, according to one aspect of the 

2 5 present invention, there is provided a remotely 

controllable apparatus which has a control panel for 
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specifying a processing operation and which is operated 
in accordance with an indication from the control panel 
and/or an externally supplied command, characterized in 
that the apparatus has storage means for storing 
5 control-panel data representing external appearance of 
at least principal portions of the control panel . 

According to another aspect of the present 
invention, there is provided a remote control method for 
controlling a remotely controllable apparatus from an 

10 external device connected to the remotely controllable 

apparatus, wherein the apparatus has a control panel for 
specifying a processing operation and which is operated 
in accordance with an indication from the control panel 
and/or an externally supplied command, the method 

15 comprising the steps of: displaying, on the external 
device, a virtual control panel having an appearance 
identical with or similar to at least part of the 
control panel; designating a desired position on the 
virtual control panel; generating a command of the 

20 remotely controllable apparatus corresponding to the 
position designated on the virtual control panel; and 
supplying the generated command to the remotely 
controllable apparatus. 

According to still another aspect of the present 

2 5 invention, there is provided a remote control system 

comprising: a remotely controllable apparatus which has 
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a control panel for specifying a processing operation 
and which is operated in accordance with an indication 
from the control panel and/or an externally supplied 
command; an information processing apparatus capable of 
5 supplying a command to the remotely controllable 

apparatus; and a display unit and input device connected 
to the information processing apparatus, wherein the 
information processing apparatus includes: display 
means for displaying, on the display unit, a virtual 

10 control panel having an appearance identical with or 
similar to at least part of the control panel; 
discriminating means for associating a command, which 
operation of the control panel causes to be applied to 
the remotely controllable apparatus, and an operation 

15 performed on the virtual control panel; and 

communication means for supplying the remotely 
controllable apparatus with a command that corresponds 
to an operation performed on the virtual control panel . 
Other features and advantages of the present 

2 0 invention will be apparent from the following 

description taken in conjunction with the accompanying 
drawings, in which like reference characters designate 
the same or similar parts throughout the figures 
thereof . 



25 



BRIEF DESCRIPTION OF THE DRAWINGS 



Fig. 1 is a block diagram illustrating an 
embodiment of the present invention; 
5 Fig. 2 is a diagram showing an example of a 

connection using a host computer; 

Fig. 3 is a diagram showing an example of the 
structure of control-panel data; 

Fig. 4 is a diagram showing the structure of 
10 simplified control-panel data and an example of the 
data ; 

Fig. 5 is a diagram showing an example of the 
transfer format of control-panel data; 

Fig. 6 is a flowchart showing a registration 
15 operation of a computer; 

Fig. 7 is a diagram showing an example of a virtual 
control panel displayed on a display unit; 

Fig. 8 is a flowchart illustrating the operation of 
the computer when a document to be transmitted has been 
2 0 placed on a facsimile/copier; 

Fig. 9 is a flowchart illustrating editing of the 
virtual control panel performed by the computer ; 

Fig. 10 illustrates an example of an editing screen 
for editing the virtual control panel; 
25 Fig. 11 is a flowchart illustrating an example of 

operation for generating a virtual control panel for 



each user; and 

Fig. 12 is a flowchart illustrating operation when 
a copier is remotely controlled using a virtual control 
panel that has been edited by the user. 

5 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 

The present invention will now be described in 
detail with reference to the drawings. Though the 
10 following embodiment is described in regard to an 

example in which a copier {referred to as a "fax/copier" 
below) having a facsimile function is used as an 
apparatus to be controlled by the user, the present 
invention is applicable other types of image processing 
15 apparatus as well. 

(Embodiment) 

Fig. 1 is a block diagram illustrating an 
embodiment of the present invention. The embodiment 
includes a computer 100 which, depending upon the form 

20 of connection, is capable of being used as a host 

computer and as a client computer. Fig. 2 illustrates 
an example in which five client computers 111 to 115 
have been connected to a fax/ copier 200 via the computer 
100 serving as a host computer. The computers 100 and 

25 111 to 115 in Fig. 2 are capable of being implemented by 
identical architectures. The computer 100 serving as 
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the host computer may also be used as a fax server. In 
such case the host computer 100 would be capable of 
extracting data received by the fax/copier 200 and of 
distributing this data to the client computers 111 - 115 
5 in conformity with subaddresses . On the client side the 
clients would be able to search in the host computer and 
retrieve the data. 

Provided in the host computer 100 are a controller 
101, a volatile memory 102, such as a RAM, which is not 

10 backed up by a power supply, a non-volatile memory 103 
such as a ROM, hard-disk drive or floppy disk, an input 
device 105 such as a keyboard and mouse, a display 
device 106 such as a CRT display or liquid crystal 
display, and an external interface 107 . 

15 The controller 101, which includes interfaces for 

the CPU, input device 105 and display device 106, 
controls the overall computer 100. 

The external interface 107, such as a 
serial/parallel interface or network interface, is for 

2 0 effecting a connection to an external device or network. 
More specifically, the external interface 107 may having 
several interfaces of various types, such as an RS-232C, 
Centronics interface, USB, SCSI, IEEE-1394 interface, 
IrDA, etc. In this embodiment, an image input device 

2 5 150 such as a scanner is connected to the host computer 
100 via the external interface 107 in addition to the 
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fax/copier 200. 

The non-volatile memory 103 is internally provided 
with an area (basic-command-data area 104) for storing 
information concerning the control panel of a remotely 
5 controlled device, as will be described later. 

The fax/copier 200 can be just a facsimile machine 
or a copier having a facsimile function (a facsimile 
machine constructed around a printer or copier) . The 
fax/copier 200 is provided with a controller 201, a 

10 volatile memory 202, a non-volatile memory 203, a 
control panel 213, a printing device 208, an image 
reader 209, a communications unit 211 and an image 
processor 212. 

The controller 201 is connected to each of the 

15 components of the fax/copier 200 and exercises overall 
control such as operation of the fax/copier 200 and 
communication with an external device. The control 
panel 213 has an input device 205 and a display unit 
206. The display unit 206 is a liquid crystal panel 

2 0 provided on the control panel, and the input device 205 
comprises hardware-type keys and buttons likewise 
provided on the control panel or a touch-sensitive panel 
superimposed on the display unit. 

The printing device 2 08 is a printer such as a 

25 laser printer or ink- jet printer. The image reader 209 
is capable of employing a CCD or contact-type image 
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sensor. The communications unit 211 comprises, e.g., a 
modem and NCU and is for communicating with an external 
communications network. The image processor 212 
comprises a gate array for converting an analog image 
5 signal read by the image reader 209 to digital data and 
subjecting the data to binarization processing, error- 
diffusion processing and encoding. In addition, the 
image processor 212 decodes a code signal received by 
the communication unit 211. The image signal obtained 

10 by decoding is printed on a printing medium by the 
printing device 2 08. 

In a case where a laser printer is used as the 
printing device 208, the latter has a semiconductor 
laser and its controller, a polygon motor and its 

15 controller, an f-0 lens, an optical-path altering 

mirror, an OPC photosensitive drum and drum driving 
unit, toner, a toner stirring unit, toner supply unit 
and its controller, a photosensitive- drum charging unit 
and its controller, a toner transfer unit, a 

20 photosensitive-drum cleaning unit and its controller, an 
image fixing unit and its controller. As these 
components are of the ordinary type, however, they are 
not described here in detail. 

The volatile memory 202 and non-volatile memory 203 

25 in the fax/copier 200 can be constructed in a manner 
similar to the corresponding memories in the host 
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computer 100. The non-volatile memory 203 has an 
internally provided control-panel data area for storing 
control-panel data 204 which includes the appearance of 
the control panel 213 and commands that correspond to 
5 the input device 205. This data is capable of being 

transmitted to the host computer 100 via the controller 
201 and an external interface 207 or to a client 
computer on a network. The structure of the control- 
panel data 204 will now be described. 

10 <Data structure> 

Fig. 3 is a diagram showing an example of the 
structure of the control-panel data 204. This example 
of the data structure illustrates a case where the 
control panel has a display unit and is provided with 11 

15 buttons (keys) . Data 1 is machine model number serving 
as ID data for identifying the model, and data 2 is 
background data indicative of the background of the 
control panel. In order to more closely approximate the 
appearance of the control panel, data relating to the 

20 background of the control panel is used as necessary. 
The background data is composed of a series of 
coordinates the origin whereof is the lower left-hand 
corner. For example, the data includes coordinate data 
indicating the position of a background border, 

25 coordinates of a reference button (e.g., a start button) 
used to determine the layout of buttons, and appearance 
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data representing background color and design. 

Data 3 is display-means data. This comprises 
coordinate data indicating a position residing on the 
display unit, a flag indicating whether or not the 
5 display unit has a touch-sensitive panel function, 
appearance data representing color and design of the 
display unit, and screen data (for four screens in this 
example) used in a case where the screen displayed on 
the display unit is changed by a command (i.e., in a 
10 case where a different instruction screen is displayed 
when a prescribed position on the display unit is 
designated) . 

Data 4 to 14 is data relating to buttons (keys) 
possessed by the control panel . Each item of this data 

15 is composed of a flag indicating whether the type is 

button or touch-sensitive panel; a title representing a 
functional explanation, such as "SET" or "CLEAR", 
indicated above or below the button on the control 
panel; the command that is generated when the button is 

2 0 pressed; relative coordinates of this button with 

respect to the reference button; button size (e.g., 
length of the size of the smallest square that contains 
the button) ; and appearance data representing the button 
color and design. 

2 5 Said command can include setting or registration 

operation, as well as, operation data for fax/ copier 
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2 00, as a computer macro. For example, the command 
consists of address setting operation to auto-dial 
button or speed dialing, copy condition setting 
{reduction/ enlargement ratio, 2-sided copying, two-page 
5 separation, etc) and operational data (perform copying, 
etc.) of a fax/copier 200. This macro command can be 
assigned to a desired button. 

In a case where background data and display-means 
data is not used and appearance data relating to each 

10 button can be prepared on the side of the computer 100, 
the amount and types of data transmitted can be reduced. 
Fig. 4 illustrates the structure of simplified control- 
panel data and example of the data. 

In Fig. 4, column Dl contains the names of control- 

15 panel buttons, and column D2 contains button- type data 
which has one-to-one correspondence with the data in 
column Dl and is indicated by a two-digit hexadecimal 
number. Columns D3 and D4 indicate the relative 
positions of the buttons, and column D5 indicates the 

20 relative sizes of the buttons. In this embodiment, the 
relative position and relative size of the start button 
serve as references, and the relative positions and 
relative sizes of the other buttons are indicated by 
two-digit hexadecimal numbers. 

25 Transfer of this control -panel data can be 

implemented by any method. For example, in a case where 
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the control-panel data having the structure shown in 
Fig. 4 is transferred using a parallel interface capable 
of transferring 32 bits, a total of 32 bits representing 
the button-type data, relative positions X, Y and 
5 relative size are sent as one block after a header is 

transmitted, as illustrated in Fig. 5. After the blocks 
of all button- type data have been sent, a footer is 
appended and sent, thereby completing the transfer. 
Depending upon the width of the interface, blocks may be 

10 split and a checksum may be provided for each block. 

Further, the data may be sent by a serial interface. It 
is also possible to send the block singly in response to 
a transmission request from the computer side. 

Further, in a case where the computer 100 is 

15 capable of preparing all of the control-panel data 
necessary for each machine model from machines in 
addition to the fax/copier 200, as described later, it 
will suffice if the control-panel data 204 is merely 
data indicative of the model. 

20 <Regi strati on operation> 

Described next will be a registration operation 
performed by the computer 100 which remotely controls 
the fax/copier 200. The registration operation is one 
in which the computer 100 creates a virtual control 

25 panel of the remotely controllable apparatus and stores 
the virtual control panel in the non-volatile memory of 



- 17 - 



the computer 100. Once the virtual control panel has 
been stored in memory, processing for displaying the 
virtual control panel can be executed the next time by 
the computer 100 alone. This raises processing speed. 
5 Fig. 6 is a flowchart illustrating the registration 

operation performed by the computer 100. The first step 
is to make initial settings, such as initializing 
communication with the fax/copier 200 (S601) . Next, 
control-panel data that has been read out of the area 

10 204 in the non-volatile memory 203 of fax/copier 200 is 
loaded into the volatile memory 102 or erasable non- 
volatile memory 103 (S602) . The loaded control-panel 
data is analyzed (S603) and, when necessary, a virtual 
control panel is created using data stored beforehand in 

15 the non-volatile memory (S604) . Correspondence is then 
established between position information concerning the 
virtual control panel and basic command data that has 
been stored in the basic-command-data area 104 of non- 
volatile memory 103 (S605) , and the correspondence data 

20 is stored in the volatile memory 102 or non-volatile 
memory 103 (S606) . 

In a case where a virtual control panel for each 
model and the correspondence data indicative of 
correspondence between the coordinates on the image and 

25 the basic commands for each model are present as a 

library in the external storage device or non-volatile 
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memory, etc., the data necessary at the time of the 
registration operation need be only information that can 
identify the control panel, such as the model name of 
the fax/copier 200, as mentioned above. Further, if it 
5 is possible for information that can identify the 
control panel to be entered directly from the input 
device of the computer 100, the registration operation 
can be concluded by the computer 100 alone. 

The library of control-panel data may be recorded 
10 collectively on so-called removable media such as a CD- 
ROM or magneto-optic disk or stored on an external 
storage device that is accessible from the computer 100. 
By way of example, the computer 100 searches the library 
on the basis of data such as the machine name entered 
15 from the input device and downloads the necessary data 
to a memory device of the computer 100. 

<Remote control> 

The operation of the computer 100 when it carries 
out remote control will now be described. In a case 

2 0 where a document that has been created by the computer 
100 is faxed by the fax/ copier 200, the virtual control 
panel of the fax/copier registered in advance by the 
above-described registration operation is displayed on 
the display device 106. Fig. 7 illustrates an example 

25 of a virtual control panel 300 displayed on the display 
device 106. Fig. 7 illustrates an example in which the 
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virtual control panel is displayed over the entire 
display area of the display device. However, an image 
of reduced size may be displayed near and edge portion, 
etc . , of the display area . 
5 Remote control is carried out by operating the 

virtual control panel as if it were the control panel of 
the actual machine. More specifically, by using a 
pointing device such as a mouse or tablet as the input 
device 105, a pointer 301 is moved over a button and 

10 clicked, or, in a case where a touch-sensitive panel is 
used as the input device, a position on the touch- 
sensitive panel corresponding to the button is pressed. 
In a case where a pointing device is not used, a desired 
key on the keyboard may be selected. 

15 The basic command that corresponds to the position 

information designated by the input device is 
transmitted to the fax/copier 200. For example, when a 
button representing a dial number 0 is clicked on the 
display device 106 (see Fig. 7) , the corresponding 

20 control-panel command data is sent to the fax/copier 

2 00, thereby issuing a command identical with that which 
would be obtained by pressing the " 0 " key on the control 
panel 213 of the fax/copier 200. 

If the transmission of commands is performed 

25 sequentially as opposed to transmission by pressing 

buttons one at a time, as described above, the commands 



may be transmitted to the fax/copier 200 collectively 
after all of the desired buttons have been pressed. 
Further, image processing at the time of a fax 
transmission may be executed in software fashion on the 
5 side of computer 100 or by the image processor 212 
within the fax/copier 200. 

Whether a command specified using the virtual 
control panel is reflected in the display area of the 
virtual control panel is optional . Though it would be 

10 desirable in terms of operability to reflect an 

operation performed on the virtual control panel in the 
display area of panel, this would increase the amount of 
control-panel data and increase the processing load 
relating to the virtual control panel. Accordingly, the 

15 choice should be made taking into consideration the 
processing capability of the computer 100. 

Further, it is not always necessary to imitate the 
appearance of the entire control panel of the actual 
machine. Only portions that are used frequently may be 

20 given an appearance close to that of the actual control 
panel, and display of buttons that are not used very 
often may be eliminated or may be replaced by images 
used commonly in all machine models. 

Furthermore, by so arranging it that the status of 

25 the fax/copier 200 is capable of being received at the 
time of remote control, the display of the virtual 
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control panel can be altered based upon the status 
information received. For example, in a case where a 
warning lamp {tally lamp) provided on the control panel 
213 of the fax/copier lights, it is possible to change 
5 the display color of the tally lamp on the virtual 
control panel using the received status. More 
specifically, an arrangement may be adopted in which the 
fax/copier 2 00 notifies the computer 100 of its status 
periodically when it is operated by remote control, and 

10 in which the computer 100 interprets the received status 
information and controls the display of the virtual 
control panel accordingly. 

transmission of document from fax/copier> 

The operation of the computer 100 when a document 

15 is transmitted using the fax/copier 200 will be 
described with reference to Fig. 8. Fig. 8 is a 
flowchart illustrating the operation of the computer 100 
when a document to be transmitted has been placed on the 
fax/ copier 200. 

20 The fax/copier 200 has an document sensor 210 (see 

Fig. 1) constituted by, e.g., a photointerrupter , that 
is capable of sensing insertion of the document to be 
transmitted. The arrangement is such that when 
insertion of the document is sensed, the document sensor 

25 210 sends the computer 100 a document- insertion signal 
indicative of the fact . 
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The computer 100 monitors signals from the external 
interface 107 constantly. If the controller 101 senses 
the document- insertion signal ("YES" at S12) , then the 
controller 101 causes the virtual control panel to be 
5 displayed on the display device 106 based upon the 

information in the non-volatile memory 103 (S13). In a 
case where the corresponding data has not been 
registered in the non-volatile memory 103, the 
registration operation described earlier is carried out 

10 or use is made of data that has already been prepared in 
an external storage device . 

Thus, remote control for setting reading 
conditions, indicating the start of reading and 
inputting the destination, etc., can be performed by the 

15 computer 100 immediately merely by inserting the 
document into the fax/ copier 200. By remotely 
controlling the fax/copier 200 from the computer 100, 
image data obtained by reading a document to be 
transmitted can be transmitted as an attachment to a 

20 word-processed document created in advance, thereby 
making unnecessary the conventional operation of 
importing the image to the word-processed document . 
<Fax transmission using image input device> 
The virtual control panel can be displayed 

25 automatically by similar processing when image data that 
has been entered using the input device 105 connected to 
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the host computer 100 is to be faxed using the 
fax/copier 2 00. 

Specifically, by adopting an arrangement in which 
when the input device 105 has sensed that a document is 
5 present on its platen or in its document feeder, the 
unit sends the computer 100 a signal indicating the 
presence of the document. This makes it possible to 
display the virtual control panel automatically in a 
manner similar to that when the fax/copier 200 is used. 

10 <Customizing the virtual control panel> 

The present invention is characterized in that the 
appearance of the virtual control panel is made to 
resemble the control panel of an actual machine. 
However, the invention also allows the layout of buttons 

15 or the like on the virtual control panel to be altered 

in accordance with user preference . An embodiment for a 
case where editing of the virtual control panel is 
performed will be described with reference to Figs. 9 
and 10. 

2 0 Fig. 9 is a flowchart illustrating editing of the 

virtual control panel performed by the computer 100, and 
Fig. 10 illustrates an example of an editing screen for 
editing the virtual control panel . 

When the virtual control panel is edited, first the 

25 position information contained in the control-panel data 
is ignored and the various control-panel elements are 
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displayed in at-a-glance form (S901) . In this 
embodiment, the elements are displayed collectively on 
the right side of the screen of the display device 106 
(see numeral 400 in Fig. 10) . Displayed in the at-a- 
5 glance grouping 400 of elements are a display area 403, 
a start button 404 and an at-a-glance grouping of 
operating keys necessary for operating the fax/copier 
2 00. As described earlier, these control buttons have 
one-to-one correspondence with the items of basic 

10 command data 104 of the fax/copier 200. For example, in 
a case where the control button 404 which indicates the 
appearance of the start button is operated, a command is 
transmitted from the computer to the fax/copier so as to 
furnish an operating effect identical with that which 

15 would be obtained by operating the start button located 
on the control panel of the fax/copier. 

Next, the border of a virtual control panel 402 is 
displayed (S902). This indicates the limits of the 
virtual control panel after editing and allows the user 

20 to grasp positional relationships when buttons are laid 
out. Though the size of the border may be fixed or made 
variable, it is preferred that a limit be set if the 
border size is made variable. 

The editing of the virtual control panel 402 can be 

25 performed by using a pointing device such as a mouse as 
an input device, and dragging desired elements, which 
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are displayed in the at-a-glance grouping 400 of 
elements, to desired positions within the border of the 
virtual control panel 402 using a pointer 401, thereby- 
placing the desired elements at the desired positions. 
5 Fig. 10 shows the button 404 being dragged into the 

virtual control panel 402 so as to be placed there as a 
button 405. 

If an element such as a button is dragged from the 
at-a-glance grouping 400 of elements ("YES" at S903) and 

10 the destination of this movement is within the border of 
the virtual control panel 402 ("YES" at S904) , then the 
computer 100 pastes the dragged element within the 
border (S905) and stores the position information in the 
volatile memory temporarily (S909) . 

15 If the end of editing is specified by designating 

an edit end button (not shown) during editing ("YES" at 
step S906) , a designation is made as to whether priority 
should be given to use of the original virtual control 
panel or the virtual control panel that has been edited 

20 (S907) . This designating information as well as the 

control-panel element information that has been stored 
temporarily in the volatile memory is stored in a 
prescribed area of the non-volatile memory (S908), after 
which editing processing is terminated. 

25 <Creation of virtual control panel on per-user 

basis, and utilization of virtual control panels> 



In the example set forth above, only one virtual 
control panel can be edited per computer. However, in 
order to deal with occasions where one computer is 
shared by a plurality of individuals, it is possible to 
5 adopt an arrangement in which a different virtual 

control panel is constructed for each individual user. 

Fig. 11 illustrates an operation for generating a 
virtual control panel for each user. The editing of the 
virtual control panel may be the same as the operation 

10 shown in Fig. 9, and the steps S901 to S908 in Fig. 9 

are executed as an editing routine (S32) . When editing 
ends, the user is prompted to enter a unique number such 
as a user ID (S3 5) . When entry of the user ID ends 
(S36) , correspondence with the edited control-panel data 

15 is established and the information is stored in the non- 
volatile memory (S3 7) . 

These items of data may be stored in a storage 
device of the host computer 100 or in a storage device 
of the fax/copier 200. In the latter case, it will be 

20 possible for a user to use his or her own edited virtual 
control panel at any computer that is capable of 
remotely controlling the fax/copier 200. On the hand, 
since all of the data will be concentrated in the 
fax/copier 2 00 in this case, a problem relating to 

25 storage capacity arises. Accordingly, where the data 
should be stored is decided upon taking these factors 



into consideration. 

An operation in which the fax/copier 200 is 
remotely controlled using a virtual control panel that 
the user him/herself has edited will be described with 
5 reference to Fig. 12. Fig. 12 is a flowchart useful in 
describing a case where the user has inserted a document 
for transmission into the image input device 105 or into 
the document feeder of the fax/copier 2 00. When the 
presence of the document is sensed by the controller 101 

10 of the host computer 100 in response to the document- 
insertion signal ("YES" at S42), the computer 100 
compares a user ID, which has been entered at step S43 
requesting input of the user ID, with data that has been 
stored in the non-volatile memory, and determines 

15 whether the corresponding virtual control panel has been 
stored (S44) . If the corresponding data exists ("YES" 
at S45) , then the computer displays the virtual control 
panel that has been edited by the user (S46) . 
Thereafter, on the basis of commands entered from the 

20 displayed virtual control panel, the user sets the 
document reading conditions, indicates start of the 
document reading operation and commands processing of 
the read image, etc. 

If the corresponding data is not found, on the 

25 other hand, then the computer transmits the user ID and 
an inquiry command to the fax/copier 200 to inquire as 
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to whether the corresponding data is present in the 
storage device of the fax/copier 200 (S47, S48) . If the 
corresponding data is found in the storage device of the 
fax/copier 2 00, the data is acquired (S49) and the user- 
5 edited virtual control panel is displayed (S46) . If the 
corresponding data is not found in the storage device of 
the fax/copier 200 either, then an inquiry as to whether 
the virtual control panel should be newly edited is sent 
to the user (S50) . If editing is desired, then an 

10 editing operation is performed in a manner similar to 

that shown in Fig. 11 (S51) , after which the user-edited 
virtual control panel is displayed (S46) . If editing 
anew is not desired, then a default virtual control 
panel possessed by the computer 100 is displayed (S52) . 

15 Adopting this arrangement makes it possible for 

each user to construct his or her own easy-to-use 
environment . 

In addition, managing address data, used for one 
button dial keys and/or speed dialing, etc., for each 
20 user ID, and if a virtual control panel of a certain 

user ID is retrieved, address data corresponding to the 
same user ID can be assigned to the buttons of the 
retrieved virtual control panel . 

Further, a batch of operations which the user 
25 frequently uses, e.g., a sequence of operations starting 
from setting a binding margin to 310mm, setting a 
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document size to A4, enabling frame erase, setting two- 
page separation mode and setting a 2 -side copy mode can 
be recorded as a macro and assigned to a certain button 
of a virtual control panel. Managing this user setting 
5 macro for each user ID allows setting of individual 
operation environment for each user. 
(Other Embodiments) 

In the foregoing embodiments, the present invention 
is described using an example in which a fax/copier is 

10 employed as the device remotely controlled by the user. 
However, if the device has a control panel and is 
capable of being remotely controlled by a user, then the 
present invention is applicable to such device. 

Further, one object of the present invention is 

15 realize a virtual control panel that is capable of being 
manipulated so that there is no disparity between a case 
where an actual machine is controlled by a user and a 
case where the machine is controlled by the user 
remotely from an information processing apparatus . It 

20 goes without saying that the scope of the present 

invention covers not only a virtual control panel having 
an appearance {positional relationship inclusive of 
background, size relationship and design) identical with 
that of the control panel of the actual machine but also 

25 a virtual control panel whose layout has been altered 

somewhat and an instance where only part of the control 
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panel of the actual machine is displayed as the virtual 
control panel . 

Furthermore, it goes without saying that the object 
of the invention is attained by supplying a storage 
5 medium storing the program codes of the software for 

performing the functions of the foregoing embodiments to 
the computer 100, reading the program codes by the 
controller 101 (e.g., a CPU or MPU) from the storage 
medium, and then executing the program codes. 

10 In this case, the program codes read from the 

storage medium implement the novel functions of the 
invention, and the storage medium storing the program 
codes constitutes the invention. 

Further, the storage medium, such as a floppy disk, 

15 hard disk, optical disk, magneto-optical disk, CD-ROM, 
CD-R, magnetic tape, non- volatile type memory card or 
ROM can be used to provide the program codes . 

Furthermore, besides the case where the aforesaid 
functions according to the embodiments are implemented 

20 by executing the program codes read by a computer, it 

goes without saying that the present invention covers a 
case where an operating system or the like running on 
the computer performs a part of or the entire process in 
accordance with the designation of program codes and 

25 implements the functions according to the embodiments. 

It goes without saying that the present invention 
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further covers a case where, after the program codes 
read from the storage medium are written in a function 
expansion board inserted into the computer or in a 
memory provided in a function expansion unit connected 
5 to the computer, a CPU or the like contained in the 
function expansion board or function expansion unit 
performs a part of or the entire process in accordance 
with the designation of program codes and implements the 
function of the above embodiment . 

10 Thus, according to the invention, as described 

above, a remotely controllable apparatus which has a 
control panel for specifying a processing operation and 
which is operated in accordance with an indication from 
the control panel and/or an externally supplied command, 

15 or an information processing apparatus that is capable 

of remotely controlling this apparatus, is provided with 
storage means for storing control-panel data that 
represents the appearance of at least principal 
components of the control panel. This makes it possible 

20 to use a virtual control panel, the appearance of which 
resembles that of the actual control panel, created by 
utilizing the control-panel data. As a result, it is 
possible to utilize the virtual control panel without 
sensing any disparity between an instance where the 

25 remotely controllable apparatus is utilized by using the 
actual control panel and an instance where the apparatus 
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is controlled remotely from an external unit. This 
affords more user-friendly operation. Since a common 
control method is adopted, the probability of erroneous 
operation by the user can be reduced. 
5 Further, my making editing of the virtual control 

panel feasible, it is possible for each user to create a 
virtual control panel that is easy for the particular 
user to use. This makes it possible to realize a remote 
control system of greater convenience. 

10 Furthermore, in a case where the remotely 

controllable apparatus is an image processing apparatus 
having a reading device for reading a document, or a 
case where an image input device for reading a document 
and generating image information has been connected to 

15 an information processing system, the insertion of a 

document or the placing of the document on a platen is 
sensed and the information processing system is so 
notified, thereby making it possible to utilize the 
virtual control panel automatically merely by inserting 

2 0 or placing the document. This reduces labor demanded of 
the user. 

As many apparently widely different embodiments of 
the present invention can be made without departing from 
the spirit and scope thereof, it is to be understood that 
25 the invention is not limited to the specific embodiments 
thereof except as defined in the appended claims. 
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WHAT IS CLAIMED IS: 

1. A data processing apparatus which has a control 
panel for specifying a processing operation and which is 
operated in accordance with an indication from the 

5 control panel and an externally supplied command, 
comprising : 

storage means for storing control-panel data 
representing external appearance of at least principal 
portions of the control panel; and 
10 transfer means for transferring the control-panel 

data that has been stored in said storage means. 

2. The apparatus according to claim 1, wherein said 
control panel has a plurality of switch means, and the 
control-panel data includes function and position 

15 information regarding at least some of said switch 
means . 

3. A remote control method for controlling a remotely 
controllable apparatus from an external device connected 
to the remotely controllable apparatus, wherein the 

20 apparatus has a control panel for specifying a 

processing operation and which is operated in accordance 
with an indication from the control panel and/or an 
externally supplied command, said method comprising the 
steps of: 

25 displaying, on the external device, a virtual 

control panel having an appearance identical with or 



- 34 - 



similar to at least part of the control panel; 

generating a command of said remotely controllable 
apparatus corresponding to an operation on said virtual 
control panel; and 
5 supplying the generated command to said remotely 

controllable apparatus. 

4. A remote control system comprising: a remotely 
controllable apparatus which has a control panel for 
specifying a processing operation and which is operated 

10 in accordance with an indication from the control panel 
and/or an externally supplied command; an information 
processing apparatus capable of supplying a command to 
said remotely controllable apparatus; and a display unit 
and input device connected to said information 

15 processing apparatus; wherein said information 
processing apparatus includes: 

display means for displaying, on the display unit, 
a virtual control panel having an appearance identical 
with or similar to at least part of the control panel; 

20 discriminating means for establishing 

correspondence between a command, which operation of the 
control panel causes to be applied to said remotely 
controllable apparatus, and an operation performed on 
the virtual control panel; and 

25 communication means for supplying said remotely 

controllable apparatus with the command corresponding to 



the operation performed on said virtual control panel . 

5. The system according to claim 4, wherein the 
operation is designation of a position. 

6. The system according to claim 4, wherein said 

5 display means displays said virtual control panel based 
upon control-panel information obtained from said 
remotely controllable apparatus. 

7. The system according to claim 4, wherein said 
display means displays said virtual control panel based 

10 upon control-panel information possessed by said 
information processing apparatus. 

8. The system according to claim 7, wherein the 
control -panel information is read out of a recording 
medium on which it has been recorded in advance and is 

15 stored in non-volatile storage means utilizable by said 
information processing apparatus. 

9. The system according to claim 4, wherein said 
remotely controllable apparatus is an image forming 
apparatus . 

20 10. The system according to claim 4, wherein said 
informing apparatus has : 

document reading means for reading a document; and 
document sensing means for sensing whether or not a 
document is present in said document reading means; 
25 wherein when said document sensing means senses 

that a document is present in said document reading 
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means, said display means displays, on said display 
unit, said virtual control panel having an appearance 
identical with or similar to at least part of the 
control panel . 

5 11. The system according to claim 4, further comprising 
an image input device connected to said information 
processing apparatus for reading the image of a document 
and supplying said information processing apparatus with 
an image signal representing the image read, said image 

10 input device having document sensing means for sensing 
whether or not the document is present; 

wherein when said document sensing means senses 
that the document is present, said display means 
displays, on said display unit, said virtual control 

15 panel having an appearance identical with or similar to 
at least part of the control .panel . 
12. The system according to claim 4, wherein 
configuration of said virtual control panel is capable 
of being altered. 

2 0 13. A storage medium storing program code of a remote 
control method for controlling a remotely controllable 
apparatus from an external device connected to the 
remotely controllable apparatus, wherein the apparatus 
has a control panel for specifying a processing 

25 operation and which is operated in accordance with an 
indication from the control panel and/or an externally 
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supplied command, comprising: 

program code for displaying, on the external 
device, a virtual control panel having an appearance 
identical with or similar to at least part of the 
5 control panel; 

program code for generating a command of said 
remotely controllable apparatus corresponding to an 
operation on said virtual control panel; and 

program code for supplying the generated command to 

10 said remotely controllable apparatus. 

14. A data processing system comprising: a data 
processing apparatus which has a control panel for 
specifying a processing operation and which is operated 
in accordance with an indication from the control panel 

15 and/or an externally supplied command; an information 
processing apparatus capable of supplying a command to 
said data processing apparatus; a display unit and input 
device connected to said information processing 
apparatus; and an image input device connected to said 

20 information processing apparatus for reading the image 
of a document and supplying said information processing 
apparatus with an image signal representing the image 
read; wherein said information processing apparatus 
includes : 

2 5 display means for displaying, on the display unit, 

a virtual control panel having an appearance identical 
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with or similar to at least part of the control panel; 

discriminating means for establishing 
correspondence between a command, which operation of the 
control panel causes to be applied to said data 
5 processing apparatus, and an operation performed on the 
virtual control panel; and 

communication means for supplying said data 
processing apparatus with the command corresponding to 
the operation performed on said virtual control panel; 
10 and 

said image input means has document sensing means 
for sensing whether or not the document is present; 

wherein when said document sensing means senses 
that the document is present, said display means 
15 displays, on said display unit, said virtual control 

panel having an appearance identical with or similar to 
at least part of the control panel . 

15. The system according to claim 14, wherein said 
display means displays, on said display unit, said 

20 virtual control panel corresponding to prescribed data 
that has been entered. 

16. The system according to claim 14, wherein said 
prescribed data is authorization data concerning each 
user who utilizes said data processing system. 

25 17. An information processing apparatus in a data 

processing system which includes: a data processing 
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apparatus which has a control panel for specifying a 
processing operation and which is operated in accordance 
with an indication from the control panel and/or an 
externally supplied command; the information processing 
5 apparatus, which is capable of supplying a command to 

said data processing apparatus; a display unit and input 
device connected to said information processing 
apparatus; and an image input device connected to said 
information processing apparatus for reading the image 

10 of a document and supplying said information processing 
apparatus with an image signal representing the image 
read; said information processing apparatus comprising: 
display means for displaying, on the display unit, 
a virtual control panel having an appearance identical 

15 with or similar to at least part of the control panel; 
discriminating means for establishing 
correspondence between a command, which operation of the 
control panel causes to be applied to said data 
processing apparatus, and an operation performed on the 

20 virtual control panel; and 

communication means for supplying said data 
processing apparatus with the command corresponding to 
the operation performed on said virtual control panel. 
18. The apparatus according to claim 16, wherein said 

25 display means displays, on said display unit, said 

virtual control panel corresponding to prescribed data 
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that has been entered. 

19. The apparatus according to claim 18, wherein said 
prescribed data is authorization data concerning each 
user who utilizes said data processing system. 
5 20. A method of controlling an information processing 
apparatus in a data processing system which includes: a 
data processing apparatus which has a control panel for 
specifying a processing operation and which is operated 
in accordance with an indication from the control panel 

10 and/or an externally supplied command; the information 
processing apparatus, which is capable of supplying a 
command to said data processing apparatus; a display 
unit and input device connected to said information 
processing apparatus; and an image input device 

15 connected to said information processing apparatus for 
reading the image of a document and supplying said 
information processing apparatus with an image signal 
representing the image read; said method comprising: 
a displaying step of displaying, on the display 

20 unit, a virtual control panel having an appearance 
identical with or similar to at least part of the 
control panel; 

a discriminating step of establishing 
correspondence between a command, which operation of the 

25 control panel causes to be applied to said data 

processing apparatus, and an operation performed on the 
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virtual control panel; and 

a communication step of supplying said data 
processing apparatus with the command corresponding to 
the operation performed on said virtual control panel. 

21. The method according to claim 20, wherein said 
display step displays, on said display unit, said 
virtual control panel corresponding to prescribed data 
that has been entered. 

22. The method according to claim 21, wherein said 
prescribed data is authorization data concerning each 
user who utilizes said data processing system. 

23 . A storage medium storing program code of a method 
of controlling an information processing apparatus in a 
data processing system which includes: a data 
processing apparatus which has a control panel for 
specifying a processing operation and which is operated 
in accordance with an indication from the control panel 
and/or an externally supplied command; the information 
processing apparatus, which is capable of supplying a 
command to said data processing apparatus; a display 
unit and input device connected to said information 
processing apparatus; and an image input device 
connected to said information processing apparatus for 
reading the image of a document and supplying said 
information processing apparatus with an image signal 
representing the image read; comprising: 
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program code for displaying, on the display unit, a 
virtual control panel having an appearance identical 
with or similar to at least part of the control panel; 

program code for establishing correspondence 
5 between a command, which operation of the control panel 
causes to be applied to said data processing apparatus, 
and an operation performed on the virtual control panel; 
and 

program code for supplying said data processing 
10 apparatus with the command corresponding to the 

operation performed on said virtual control panel. 
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ABSTRACT OF THE DISCLOSURE 



Disclosed is a remotely controllable apparatus and 
a remote control method in which there is little 
5 disparity between a case where the apparatus is 
controlled using a control panel possessed by the 
apparatus itself and a case where the apparatus is 
controlled remotely from a personal computer or the 
like, thereby facilitating control of the apparatus. A 

10 remotely controllable apparatus which has a control 

panel for specifying a processing operation and which is 
operated in accordance with an indication from the 
control panel and/or an externally supplied command, or 
an information processing apparatus that is capable of 

15 remotely controlling this apparatus, is provided with 
storage device for storing control -panel data that 
represents the appearance of at least principal 
components of the control panel. Remote control is 
performed using a virtual control panel, the appearance 

20 of which resembles that of the actual control panel, 
created by utilizing the control-panel data. 
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FIG. 8 
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FIG. 11 
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FIG. 12 
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As a below named inventor, I hereby declare that: 

My residence, post office address and citizenship are as stated below next to my name; 



I believe I am the original, first and sole inventor (if only one name is listed below) or an original, first and joint inventor (if plural 
names are listed below) of the subject matter which is claimed and for which a patent is sought on the invention entitled USER 

INTERFACE DISPLAY APPARATUS AND METHOD OF CONTROLLING SAME 



the specification of which | X [ is attached hereto | | was filed on as United States Application 

No. or PCT International Application No. 

and was amended on (if applicable). 



I hereby state that I have reviewed and understand the contents of the above-identified specification, including the claims, as 
amended by any amendment referred to above. 



I acknowledge the duty to disclose information which is material to patentability as defined in 37 CFR §1 .56. 



I hereby claim foreign priority benefits under 35 U.S.C. §119(a)-(d) or §365(b), of any foreign application^) for patent or 
inventor's certificate, or § 365(a) of any PCT international application which designates at least one country other than the United States, 
listed below and have also identified below any foreign application for patent or inventor's certificate, or PCT international application 
having a filing date before that of the application on which priority is claimed: 

(Yes/No) 

Country Application No. Piled (Dav/Mo./Yr.) Priority Claimed 

Japan 10-250015 3/September/1998 Yes 



I hereby claim the benefit under 35 U.S.C. § 120 of any United States application(s), or § 365(c) of any PCT international 
application designating the United States, listed below and, insofar as the subject matter of each of the claims of this application is not 
disclosed in the prior United States or PCT international application in the manner provided by the first paragraph of 35 U.S.C. § 112, 
I acknowledge the duty to disclose information which is material to patentability as defined in 37 C.F.R. § 1.56 which became available 
between the filing date of the prior application and the national or PCT international filing date of this application. 

Status 

Application No. Filed (Day/Mo ./Yr.) (Patented. Pending. Abandoned) 



I hereby appoint the practitioners associated with the firm and Customer Number provided below to prosecute this application 
and to transact all business in the Patent and Trademark Office connected therewith, and direct that all correspondence be addressed to 
the address associated with that Customer Number: 

FITZPATRICK, CELLA, HARPER & SCINTO 
Customer Number: 05514 



I hereby declare that all statements made herein of my own knowledge are true and that all statements made on information and 
belief are believed to be true; and further that these statements were made with the knowledge that willful false statements and the like 
so made are punishable by fine or imprisonment, or both, under Section 1001 of Title 18 of the United States Code and that such 
willful false statements may jeopardize the validity of the application or any patent issued thereon. 

Full Name of Sole or First Inventor MAS AH I KQ YOKOTA 

Inventor's signature 

Date Citizen/Subject of JAPAN 

Residence 20-10, Tennodai 6-chome. Abiko-shi, Chiba-ken, Japan 



Post office Address c/o CANON KABUSHIKI KAISHA. 30-2, Shimomaruko 
3-chome, Ohta-ku. Tokvo, Japan 



